Method of effectively predicting multi-layer based video frame, and video coding method and apparatus using the same

ABSTRACT

A method and apparatus are provided for effectively predicting a video frame that use all of the advantages of an intra mode and an intra BL mode in multi-layer structure based-video coding. The method includes reconstructing the intra block of a lower layer using the previously reconstructed first neighboring blocks of the intra block; subtracting the first neighboring blocks from previously stored the second neighboring blocks of an upper layer corresponding to the first neighboring blocks; creating a differential predicted block based on a predetermined intra prediction mode by performing intra prediction using virtual differential neighboring blocks that are created as a result of the subtraction; adding the differential predicted block and the reconstructed intra block; and subtracting a predicted block, which is created as a result of the addition, from a block of the upper layer corresponding to the intra block.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Korean Patent Application No.10-2005-0016270 filed on Feb. 26, 2005 in the Korean IntellectualProperty Office, and U.S. Provisional Patent Application No. 60/646,534filed on Jan. 25, 2005 in the United States Patent and Trademark Office,the disclosures of which are incorporated herein by reference in theirentirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Apparatuses and methods consistent with the present invention relategenerally to video coding, and more particularly, to effectivelypredicting a video frame that use all of the advantages of an intra modeand an intra BL mode in multi-layer structure based-video coding.

2. Description of the Related Art

As information and communication technology, including the Internet,develops, image-based communication as well as text-based communicationand voice-based communication is increasing. The existing text-basedcommunication is insufficient to satisfy various consumers demands.Therefore, the provision of multimedia service capable of accommodatingvarious types of information, such as text, images and music, isincreasing. Since the amount of multimedia data is large, multimediadata require high-capacity storage media and require broad bandwidth atthe time of transmission. Therefore, to transmit multimedia data,including text, images and audio, it is essential to use a compressioncoding technique.

The fundamental principle of data compression is to eliminate redundancyin data. Data can be compressed by eliminating spatial redundancy suchas the case where an identical color or object is repeated in an image,temporal redundancy such as the case where there is little changebetween neighboring frames or identical audio sound is repeated, orpsychovisual redundancy in which the fact that humans' visual andperceptual abilities are insensitive to high frequencies is taken intoaccount.

For such a moving image compression method, H.264/Advanced Video Coding(AVC), which has higher compression efficiency than Moving PictureExperts Group (MPEG)-4, has attracted attention recently. H.264 usesdirectional intra-prediction, which eliminates spatial similarity ineach frame, as one of the schemes for improving compression efficiency.

Directional intra-prediction is a method of predicting the values of acurrent sub-block and encoding only a difference in such a way as toperform copying in predetermined directions using neighboring pixels onthe upper and left sides of a sub-pixel with respect to the sub-block.

In H.264, a predicted block with respect to a current block is generatedbased on other blocks having preceding sequential positions. Thedifference between the current block and the predicted block is encoded.For a luminance component, each predicted block is generated on a 4×4block or 6×16 macroblock basis. There are a total of nine optionalprediction modes for each 4×4 block, whereas there are a total of fouroptional prediction modes for each 16×16 block. An H.264-based videoencoder selects the one prediction mode that minimizes the differencebetween the current block and the predicted block, from among theprediction modes, for each block.

For prediction modes for the 4×4 block, H.264, as shown in FIG. 1,employs a total of nine prediction modes, including a total of ninedirectional modes (modes 0, 1, 3 to 8), and a DC mode (mode 2) that usesan average of the values of nine neighboring pixels.

FIG. 2 illustrates an example of labeling to illustrate the nineprediction modes. In this example, a predicted block (including regions“a” to “p”) with respect to a current block is generated usingpreviously decoded samples A to M. If regions E, F, G and H cannot bepreviously decoded, regions E, F, G and H can be virtually created bycopying region D to the locations of the regions E, F, G and H.

With reference to FIG. 3, the nine prediction modes are respectivelydescribed in detail below. In the case of mode 0, the pixels of apredicted block are extrapolated using upper samples A, B, C and D in avertical direction, and in the case of mode 1, the pixels areextrapolated using left samples I, J, K and L in a horizontal direction.Furthermore, in the case of mode 2, the pixels of the predicted blockare uniformly replaced by the averages of upper samples A, B, C and Dand left samples I, J, K and L.

In the case of mode 3, the pixels of the predicted block areinterpolated between a lower-left position and an upper-right positionin a direction that is inclined at an angle of 45°, and in the case ofmode 4, the pixels are extrapolated in a direction that is inclinedtoward an upper-left position at an angle of 45°. Furthermore, in thecase of mode 5, the pixels of the predicted block are extrapolated in adirection that is inclined rightward from a vertical direction at anangle of about 26.6° (width/height=1/2).

In the case of mode 6, the pixels of the predicted block areextrapolated in a direction that is inclined downward from a horizontaldirection at an angle of about 26.6°, and in the case of mode 7, thepixels are extrapolated in a direction that is inclined leftward from avertical direction at an angle of about 26.6°. Finally, in the case ofmode 8, the pixels of the predicted block are interpolated in adirection that is inclined upward from a horizontal direction at anangle of about 26.6°.

The arrows of FIG. 3 indicate prediction directions in respective modes.In modes 3 to 8, the samples of the predicted block can be generatedfrom the weighted averages of previously decoded reference samples A toM. For example, in the case of mode 4, sample d, which is located in theupper left, can be predicted as expressed by the following Equation 1.In this Equation, the round(.) function is a function that rounds off aninput value to an integer position.d=round(B/4+C/2+D/4)  (1)

Meanwhile, a 16×16 prediction model for luminance components includesfour modes, that is, mode 0, mode 1, mode 2 and mode 3. In the case ofmode 0, the pixels of a predicted block are extrapolated from uppersamples H, and in the case of mode 1, the pixels of a predicted blockare extrapolated from left samples V. Furthermore, in the case of mode2, the pixels of a predicted block are calculated using the averages ofupper samples H and left samples V. Finally, in the case of mode 3, a“plane” function suitable for upper samples H and left samples V isused. This mode is more suitable for a region in which luminancesmoothly changes.

Meanwhile, in addition to efforts to improve the efficiency of videocoding, research into video coding that allows the resolution oftransmitted video data, a frame rate, and a Signal-to-Noise Ratio (SNR)to be adjusted, that is, that supports scalability, is actively beingcarried out.

With regard to this scalable video coding technique, standardizationwork is in progress in the Moving Picture Experts Group (MPEG)-21PART-13. Of these methods for supporting scalability, a multi-layeredvideo coding method is considered a prominent method. For example,multiple layers, including a base layer, a first enhanced layer and asecond enhanced layer 2, are provided and respective layers havedifferent resolutions QCIF, CIF and 2CIF or different frame rates.

In the scalable video coding standard currently in progress, besidesinter prediction and directional intra prediction (hereinafter simplyreferred to as intra prediction) used in existing H.264 to predict acurrent block or macroblock, a method of predicting a layer to which acurrent block belongs using the correlation between the current blockand a corresponding lower layer block is additionally introduced. Thisprediction method is referred to as “intra BL (intra BL) prediction” inthe standard, and the case of performing encoding using such predictionis referred to as “intra BL mode.”

FIG. 4 is a schematic diagram showing the three prediction methods,which illustrates the case of performing intra prediction on amacroblock of a current frame 1 ({circle around (1)}), the case ofperforming inter prediction using a frame 2 placed at a temporallocation different from that of the current frame 1 ({circle around(2)}), and the case of performing intra BL prediction using texture dataabout the region 6 of the frame of a base layer corresponding to amacroblock ({circle around (3)}).

As described above, in the scalable video coding standard, theadvantageous one of the three prediction methods is selected for eachmacroblock, and a corresponding macroblock is encoded using the selectedmethod. That is, for one macroblock, inter prediction, intra predictionand intra BL prediction are selectively used. However, a differentialblock created using intra BL prediction still has considerablecorrelation with neighboring differences. Accordingly, it is necessaryto develop a prediction technique that takes the advantages of bothintra BL prediction and intra prediction into account. Although aprediction method that takes the advantages of intra prediction, intraBL prediction and inter prediction into account may be considered, thecharacteristics of intra BL prediction and intra prediction areconsiderably different from those of inter prediction, so this method isnot desirable.

SUMMARY OF THE INVENTION

The present invention provides a prediction method that uses both intraBL prediction and intra prediction.

Furthermore, the present invention may improve video coding efficiencyusing the prediction method.

According to an aspect of the present invention, there is provided amethod of efficiently predicting a multi-layer based video frame,including reconstructing the intra block of a lower layer using thepreviously reconstructed first neighboring blocks of the intra block;subtracting the first neighboring blocks from previously stored thesecond neighboring blocks of an upper layer corresponding to the firstneighboring blocks; creating a differential predicted block based on apredetermined intra prediction mode by performing intra prediction usingvirtual differential neighboring blocks that are created as a result ofthe subtraction; adding the differential predicted block and thereconstructed intra block; and subtracting a predicted block, which iscreated as a result of the addition, from a block of the upper layercorresponding to the intra block.

According to an aspect of the present invention, there is provided amethod of efficiently predicting a multi-layer based video frame,including reconstructing the intra block of a lower layer usingpreviously reconstructed the first neighboring blocks of the intrablock; subtracting the first neighboring blocks from the previouslystored second neighboring blocks of an upper layer corresponding to thefirst neighboring blocks; creating a differential predicted block basedon a predetermined intra prediction mode by performing intra predictionusing virtual differential neighboring blocks that are created as aresult of the subtraction; adding the differential predicted block andthe reconstructed intra block; subtracting a predicted block, which iscreated as a result of the addition, from a block of the upper layercorresponding to the intra block; and encoding a residual block that iscreated as a result of the subtraction.

According to an aspect of the present invention, there is provided amethod of efficiently predicting a multi-layer based video frame,including reconstructing the intra block of a lower layer using thepreviously reconstructed first neighboring blocks of the intra block;subtracting the first neighboring blocks from the previously storedsecond neighboring blocks of an upper layer corresponding to the firstneighboring blocks; creating a differential predicted block based on apredetermined intra prediction mode by performing intra prediction usingvirtual differential neighboring blocks that are created as a result ofthe subtraction; adding the differential predicted block and thereconstructed intra block; reconstructing a block of the upper layercorresponding to the intra block; and adding the reconstructed block ofthe upper layer and a predicted block that is acquired as a result ofthe addition.

According to an aspect of the present invention, there is provided amulti-layer based video encoder, including a means for reconstructingthe intra block of a lower layer using the previously reconstructedfirst neighboring blocks of the intra block; a means for subtracting thefirst neighboring blocks from the previously stored second neighboringblocks of an upper layer corresponding to the first neighboring blocks;a means for creating a differential predicted block based on apredetermined intra prediction mode by performing intra prediction usingvirtual differential neighboring blocks that are created as a result ofthe subtraction; a means for adding the differential predicted block andthe reconstructed intra block; a means for subtracting a predictedblock, which is created as a result of the addition, from a block of theupper layer corresponding to the intra block; and a means for encoding aresidual block that is created as a result of the subtraction.

According to an aspect of the present invention, there is provided amulti-layer based video encoder, including a means for reconstructingthe intra block of a lower layer using the previously reconstructedfirst neighboring blocks of the intra block; a means for subtracting thefirst neighboring blocks from the previously stored second neighboringblocks of an upper layer corresponding to the first neighboring blocks;a means for creating a differential predicted block based on apredetermined intra prediction mode by performing intra prediction usingvirtual differential neighboring blocks that are created as a result ofthe subtraction; a means for adding the differential predicted block andthe reconstructed intra block; a means for reconstructing a block of theupper layer corresponding to the intra block; and a means for adding thereconstructed block of the upper layer and a predicted block that isacquired as a result of the addition.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects of the present invention will be moreclearly understood from the following detailed description of theexemplary embodiments taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a diagram illustrating prediction directions for respectivemodes defined by H.264;

FIG. 2 is a diagram illustrating an example of labeling that is used toillustrate the intra prediction modes of FIG. 1;

FIG. 3 is a diagram illustrating the intra prediction modes of FIG. 1 indetail;

FIG. 4 is a schematic diagram illustrating conventional three predictionmethods;

FIG. 5 is a diagram illustrating the concept of a prediction methodaccording to an exemplary embodiment of the present invention;

FIG. 6 is a block diagram illustrating the construction of a videoencoder according to an exemplary embodiment of the present invention;

FIG. 7 is a block diagram illustrating the schematic construction of anencoding unit that is included in the video encoder of FIG. 6;

FIG. 8 is a block diagram illustrating the schematic construction of adecoding unit that is included in the video encoder of FIG. 6;

FIG. 9 is a block diagram illustrating the construction of a videoencoder according to another exemplary embodiment of the presentinvention;

FIG. 10 is a block diagram illustrating the construction of a videoencoder according to still another exemplary embodiment of the presentinvention;

FIG. 11 is a view illustrating neighboring direction with respect to avertical mode;

FIG. 12 is a view illustrating neighboring directions with respect to 8intra prediction modes having directionality;

FIG. 13 is a diagram illustrating the corresponding regions betweenlayers when the resolutions of the layers do not coincide with eachother; and

FIG. 14 is a block diagram illustrating the construction of a videodecoder according to an exemplary embodiment of the present invention.

DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Reference now should be made to the drawings, in which the samereference numerals are used throughout the different drawings todesignate the same or similar components.

FIG. 5 is a diagram illustrating the concept of a prediction methodaccording to an exemplary embodiment of the present invention. In themethod, how to create a predicted block 40 for the original image of acurrent block 10 (hereinafter simply referred to as a current block)that belongs to an upper layer is an essential point. In the presentinvention, the term “block” may be defined as a block having a sizeidentical to that of an intra block of the H.264 standard. Meanwhile,for a conventional intra block, 4×4 mode and 16×16 mode exist forluminance components and 8×8 mode exists for chrominance components and,accordingly, the “block” of the present invention may have one of thevarious sizes.

In an upper layer, before the prediction of a current block 10, theneighboring blocks of the current block 10 have already beenencoded/decoded using a certain prediction method. The neighboringblocks 15 may be encoded/decoded using any prediction method, such asinter-prediction, intra-prediction, or intra BL prediction.

In order to predict the current block 10 as described above, imagespassed through encoding and decoding (closed loop), rather than theoriginal images of the neighboring blocks 15, are used. Currently, mostcodecs eliminate drifting error, which is produced between a videoencoder and a video decoder, using such closed loop encoding, so thatreconstructed images, which are decoded after being encoded, are used asthe images of existing blocks that are used to predict a current blockin the present invention. However, it is apparent to those skilled inthe art that the present invention can also be applied to an open loopmethod using the original images of existing blocks.

In a lower layer, a block corresponding to the current block 10(hereinafter referred to as a “corresponding block”) refers to an intrablock 20 that has been encoded and decoded using intra prediction asshown in FIG. 3. Furthermore, the neighboring blocks 25 of the intrablock 20 can be encoded/decoded using any prediction method in the samemanner as the neighboring blocks 15 of the upper layer. The neighboringblocks 25 of the lower layer exist at locations corresponding to thoseof the neighboring blocks 15.

Meanwhile, the resolution of the upper layer may be identical to ordifferent from the resolution of the lower layer. If the resolutions ofboth layers are identical to each other, the size of the blocks 10 and15 of the upper layer is identical to the size of the blocks 20 and 25of the lower layer. Otherwise, the blocks 10 and 15 of the upper layermay be larger than those of the lower layer. For example, if theresolution of the upper layer is two times the resolution of the lowerlayer and the blocks 20 and 25 of the lower layer have a 4×4 size, theblocks of the upper layer may have an 8×8 size.

Furthermore, although the present invention is described with the term“neighboring block” referring to four blocks, that is, left, upper left,upper and upper right blocks, which are referred to for intraprediction, the number and locations of neighboring blocks may vary ifthe type of blocks, which are referred to for intra prediction, varies.

As described above, when the neighboring blocks 15 of the upper layerand the intra block 20 and neighboring blocks 25 of the lower layer havebeen all reconstructed, the predicted block 40 of the current block 10can be created using them. A process of creating the predicted block 40is described in detail below.

First, virtual differential neighboring blocks 35 are created bysubtracting the reconstructed neighboring blocks 25 of the lower layerfrom the neighboring blocks 15 of the upper layer. The subtraction isperformed between the corresponding blocks of both layers. Whendirectional intra prediction illustrated in FIG. 3 is applied to theneighboring blocks 35 created as described above, differential predictedblocks 30 can be created for nine modes.

Thereafter, each of the nine created differential predicted blocks 30 isadded to the intra block 20, and then an optimal candidate is selectedfrom nine candidates obtained for the predicted block 40. For example,the selection may be performed in such a way as to select one thatminimizes the image different between each of the nine candidates andthe current block 10. Alternatively, for another example, a method ofpredicting the current block 10 using the nine candidates for thepredicted block 40, encoding prediction results and selecting an optimalcandidate using the Rate-Distortion (R-D) cost function may be used. Theformer method has the advantages of a small amount of calculation andsimple performance, while the latter method has the advantage of moreaccurate selection between candidates.

When the predicted block 40 is selected from the candidates for thepredicted block 40, the current block 10 is predicted using thepredicted block 40. That is, a residual block 50 is created bysubtracting the predicted block 40 from the created current block 10.Thereafter, the fundamental operation of the present invention iscompleted by encoding the residual block 50.

The schematic operation of the present invention is described above. Theconventional method using an intra BL mode is schematically compared tothe method of the present invention from a conceptual point of view.When the image of the current block 10 is C1 and the reconstructed intrablock 20 of the lower layer is C0, C0 is used as the predicted block ofC1 in the conventional intra BL mode on the assumption that C0 issimilar to C1, as shown in Equation 2.C1−C0≅0  (2)

In contrast, the present invention does not use C0 as the predictedblock, but instead uses the correlation of the following Equation 3. InEquation 3, INT(.) refers to a function that performs directional intraprediction, N1 refers to the neighboring blocks of the upper layer andN₀ refers to the neighboring blocks of the lower layer.C₁−C₀≅INT(N₁−N₀)≠0  (3)

As shown in Equation 3, the difference between the current block C₀ andthe corresponding lower layer block C₁ has considerable correlation withthe results of intra prediction that is performed using the neighboringblocks N₀ of the upper layer and the neighboring blocks N₀ of the lowerlayer. Accordingly, the current block C₁ based on the present inventioncan be more accurately predicted using the following Equation 4.C ₁ ≅C ₀ +INT(N ₁ −N ₀)  (4)where C₁, C₀, and INT(N₁−N₀) designate the current block 10,reconstructed intra block 20 and differential predicted block 30 of FIG.5, respectively.

FIG. 6 is a block diagram showing the construction of a video encoder100 according to an exemplary embodiment of the present invention. Thevideo encoder 100 is constructed to use current and neighboring blockimages and the images of the corresponding block and its neighboringblocks of the lower layer as input and to output data in which thecurrent block is encoded.

An encoding unit 105 encodes the images of the neighboring blocks of acurrent block, and a decoding unit 110 decodes encoded results. Theencoding and decoding processes follow general video encoding/decodingprocesses. The encoding unit 105 may be represented by the schematicconstruction of FIG. 7. For the input neighboring block images, onemethod is selected from among inter prediction, intra prediction andintra BL prediction by a selection unit 61. In conformity with theselected prediction method, one of an inter prediction unit 62, an intraprediction unit 63 and an inter BL prediction unit 64 creates a residualblock from the input images. A transform unit 65 creates a transformcoefficient from the residual block using a transform algorithm, such asdiscrete cosine transform (DCT), or wavelet transform, and aquantization unit 66 quantizes the transform coefficient.

Furthermore, the decoding unit 110 may be represented by the schematicconstruction of FIG. 8. An inverse quantization unit 71 inverselyquantizes signals output from the encoding unit 105, and an inversetransform unit 72 performs the transform process of the transform unit65 on inversely quantized results in inverse order. Furthermore, aselection unit 73 selects an inverse prediction method corresponding tothe prediction method selected by the encoding unit 105. In conformitywith the selection, an inverse inter prediction unit 74, an inverseintra prediction unit 75, or an inverse intra BL prediction unit 76reconstructs neighboring blocks by performing the inverse of the processof the encoding unit 105, that is, inverse prediction, on inverselytransformed signals. The reconstructed neighboring blocks are input to asubtractor 135.

An intra encoding unit 115 encodes the corresponding block image, andthe intra decoding unit 120 decodes encoded results. It should be notedthat in the present invention, the corresponding block must be encodedusing intra prediction. However, the neighboring blocks of the currentblock or the neighboring blocks of the corresponding block may beencoded using any prediction method.

Accordingly, the intra encoding unit 115 may be formed of the intraprediction unit 63, the transform unit 65 and the quantization unit 66,as shown in the block diagram of FIG. 7, and the intra decoding unit 120may be formed of the inverse quantization unit 71, the inverse transformunit 72 and the inverse intra prediction unit 75, as shown in the blockdiagram of FIG. 8.

A signal output from the intra decoding unit 120, that is, areconstructed corresponding block (intra block), is input to an adder160 selectively through an up-sampler 150. The up-sampler 150 is notused when the resolutions of the upper layer and the lower layercoincide with each other, and performs up-sampling so that theresolution of the reconstructed intra block coincides with that of theupper layer when the resolutions do not coincide with each other.

Meanwhile, the encoding unit 125 encodes the images of the neighboringblocks of the corresponding block, and the decoding unit 130 decodes theencoded results. The encoding unit 125 may have a construction identicalto that of FIG. 7, and the decoding unit 130 may have a constructionidentical to that of FIG. 8.

A signal output from the decoding unit 130, that is, reconstructedneighboring blocks, is input to the subtractor 160 selectively throughthe up-sampler 140. The up-sampler 140 is not used when the resolutionsof the upper layer and the lower layer coincide with each other, andperforms up-sampling so that the resolution of the reconstructedneighboring blocks coincides with that of the upper layer when theresolutions do not coincide with each other.

The subtractor 135 acquires differential neighboring blocks bysubtracting a signal, which is input from the decoding unit 130 orup-sampler 140, from a signal, which is input from the decoding unit110.

A predicted block creation unit 145 creates a predetermined number ofdifferential predicted blocks for respective modes by performingdirectional intra prediction using the differential neighboring blocks.For example, as shown in FIG. 3, in H.264, a total of nine modes,including eight modes having directionality and one DC mode, can beused. Nine differential predicted blocks are created for the respectivemodes, and the created differential predicted blocks are provided to theadder 160.

The adder 160 adds the differential predicted blocks for the respectivemodes to the signal input from the intra decoding unit 120 or from theup-sampler 150. As a result, a number of the predicted block candidatesequal to the number of modes are created.

The mode selection unit 190 selects an optimal mode (intra predictionmode) from the modes that the predicted block candidates have, andselects an optimal predicted block from the predicted block candidatesbased on the selected mode. The mode selection is performed in such away as to select the mode that minimizes the difference between thepredicted block candidates and the current block, as described above. Inthis case, the difference between the blocks refers to the sum of thedifferences between corresponding pixel values of the blocks.

Thereafter, the subtractor 165 creates a residual block by subtractingthe selected predicted block from the image of the input current block.

The transform unit 170 creates a transform coefficient performingspatial transform on the residual block. As a spatial transform method,DCT, wavelet transform, etc. may be used. If DCT is used as the spatialtransform, the transform coefficient refers to a DCT coefficient. Whenwavelet transform is used as the spatial transform, the transformcoefficient refers to a wavelet coefficient.

The quantization unit 175 quantizes the transform coefficient. The termquantization refers to a process of dividing the transform coefficient,which is represented by arbitrary real values, at regular intervals,representing the transform coefficient with discrete values, andmatching the discrete values to predetermined indices. In particular,when wavelet transform is used for the spatial transform, an embeddedquantization method is widely used as the quantization method. Theembedded quantization method includes the Embedded Zerotrees WaveletAlgorithm (EZW), Set Partitioning in Hierarchical Trees (SPIHT),Embedded ZeroBlock Coding (EZBC), or other methods known to thoseskilled in the art.

The entropy encoding unit 180 encodes the transform coefficient, whichis quantized by the quantization unit 175, and information about themode, which is selected by the mode selection unit 190, without loss. Asa lossless encoding method, arithmetic coding, variable length coding,etc. may be used, with the result that the encoded data of the currentblock are output.

FIG. 9 is a block diagram illustrating the construction of a videoencoder according to another exemplary embodiment of the presentinvention. The exemplary embodiment of FIG. 9 employs the scheme inwhich the mode selection unit 185 uses results losslessly encoded by theentropy encoding unit 180, unlike the exemplary embodiment of FIG. 6.Accordingly, the differential predicted block creation unit 145 createsdifferential predicted blocks for respective modes, for example, 9 intraprediction modes, and creates predicted blocks, residual blocks,transform coefficients and quantized coefficients for respective modes,and an optimal mode is finally selected through comparison in the modeselection unit 185.

The mode selection unit 185 applies the R-D cost function to the dataencoded for respective modes by the entropy encoding unit 180, andselects the mode that minimizes the R-D cost function. The R-D costfunction may be defined by the following Equation 5. In this Equation, Erefers to the difference between a signal (reconstructed current data),in which the encoded data are reconstructed, and an original signal(current block), and B refers to the number of bits that is required toperform each method. Furthermore, λ is a Lagrangian coefficient, andrefers to the coefficient that can adjust the reflection ratios of E andB.C=E+λB  (5)

The mode selected by the mode selection unit 185 is transferred to theentropy encoding unit 180, and the entropy encoding unit 180 encodesinformation about the selected mode without loss, and outputs encodedinformation along with the encoded data of the current blockcorresponding to the selected mode.

The exemplary method proposed in FIGS. 6 and 9 requires a large numberof symbol bits compared to the existing intra BL prediction. The reasonfor this is that the existing intra BL prediction does not requireadditional mode information while the prediction method according to thepresent invention requires 9 pieces of additional mode information. Theoverhead based on the directional intra prediction modes can beeliminated using intra prediction mode on the base layer as it is.

FIG. 10 is a block diagram showing the construction of the video encoder400 in the case of using the intra prediction mode, which is used at thetime of intra prediction in the lower layer, so as to create thedifferential predicted block (30 of FIG. 5) from the differentialneighboring blocks (35 of FIG. 5).

Most of the construction of FIG. 10 is similar to the construction ofFIG. 6 or 9. The construction of FIG. 10 is different from theconstruction of FIG. 6 or 9 in that information about the mode, which isprovided by the intra encoding unit 115, is input to the differentialpredicted block creation unit 145 and the entropy encoding unit 180. Thedifferential predicted block creation unit 145 creates a singledifferential predicted block based on the information about the mode.The differential predicted block creation unit 145 creates a number ofdifferential predicted blocks equal to the number of modes in theexemplary embodiments of FIGS. 6 and 9.

Meanwhile, the entropy encoding unit 180 does not need to encodeinformation about the mode of the current block, but encodes onlyinformation about the mode of the corresponding block of the lowerlayer. Accordingly, the present invention does not increase overheadcompared to existing intra BL prediction.

Although, in the exemplary embodiments of FIGS. 6, 9, and 10, theprocess of encoding the current block in accordance with the presentinvention has been described, those skilled in the art can easilyappreciate that a video frame can be encoded by combining the encodedcurrent block with the previously encoded neighboring blocks.

Meanwhile, when, like the exemplary embodiments of FIGS. 6 and 9,additional prediction modes are calculated, it is necessary to develop amethod of somewhat reducing overhead caused by the additional predictionmodes. Accordingly, in the present invention, a method of encoding onlydirectional differences with respect to the prediction mode of thecorresponding block, not the information about the mode of the currentblock, is used.

FIG. 11 is a diagram illustrating a method of performing searches onlyin directions neighboring a vertical direction in the current block whenthe optimal prediction direction of the corresponding block is avertical mode (mode 0). That is, since the optimal prediction mode ofthe corresponding block represents the vertical direction, there is astrong possibility that the optimal prediction mode of the current blockis a vertical mode (mode 0), a vertical left mode (mode 7), or avertical right mode (mode 5). Accordingly, by searching only modescorresponding to the directions, the amount of calculation at the timeof directional intra prediction can be reduced. Additionally, byrepresenting a clockwise neighboring direction with −1, acounterclockwise neighboring direction with +1, and the same directionwith 0 and encoding them, the number of bits required to encode optimaldirections can be effectively reduced.

As described above, each mode can be represented with a differenceregardless of a mode number, with only the direction thereof being takeninto account. In the present invention, this difference is defined as a“directional difference.” For example, on the basis of mode 0, thedirectional difference of mode 6 is +3, and the directional differenceof mode 3 is −2.

FIG. 12 is a diagram illustrating neighboring directions with respect to8 intra prediction modes having directionality. For example, theneighboring modes of mode 7 are mode 3 and mode 0, and the neighboringmodes of mode 0 are mode 7 and mode 5. There is the problem of how todetermine the neighboring modes of mode 3 and mode 8. In an exemplaryembodiment of the present invention, neighboring modes may be defined astwo modes that are most adjacent to a mode in clockwise andcounterclockwise directions regardless of a distance thereto.Accordingly, the neighboring modes of mode 3 are mode 8 and mode 7, andthe neighboring modes of mode 8 are mode 1 and mode 3. Based on theabove description, the neighboring modes of a specific mode arerepresented with −1 or 1, and uniformity is assured for all the intraprediction modes having directionality.

However, modes 3 and 8 represent almost opposite directions, so that itis difficult to regard each of the modes 3 and 8 as falling within theother's prediction range. Therefore, in another exemplary embodiment ofthe present invention, modes 3 and 8 may each have a single neighboringmode. In this case, the neighboring mode of mode 3 is mode 7 and theneighboring mode of mode 8 is mode 1.

Meanwhile, when the resolution of the current layer is different fromthe resolution of the lower layer, the current block and the lower layerblock do not correspond to each other on a one to one basis. Referringto the example of FIG. 13, if the resolution of the lower layer is ½ ofthat of the current layer, a single block of the lower layer correspondsto four blocks 81 to 84. Accordingly, in this case, it should be notedthat the blocks of a lower layer corresponding to the four blocks 81 and84 of the current block are all a block 85.

By representing the modes of the current block with directionaldifferences and entropy-encoding the directional differences, overheadcan be somewhat reduced.

FIG. 14 is a block diagram illustrating the construction of the videodecoder 200 according to an exemplary embodiment of the presentinvention.

An entropy decoding unit 205 decodes an input bit-stream without loss,and extracts data on the neighboring blocks of a current block, data ona corresponding block, data on the neighboring blocks of thecorresponding block, data on the current block and information about themode of the current block.

The decoding unit 210 decodes data on the neighboring blocks of thecurrent block and provides decoding results to a subtractor 230. Thedecoding unit 210 may be constructed to be the same as the decoding unit110 of FIG. 6. Meanwhile, the intra decoding unit 215 decodes data onthe corresponding block and provided decoding results to an adder 240.The intra decoding unit 215 may be constructed to be the same as theintra decoding unit 120 of FIG. 6.

Furthermore, the decoding unit 220 decodes data on the neighboringblocks of the corresponding block and provides decoding results to thesubtractor 230 a selectively through an up-sampler 225. The decodingunit 220 may be constructed to be the same as the decoding unit 130 ofFIG. 6. The up-sampler 225 is not used when the resolution of an upperlayer coincides with the resolution of a lower layer, and performsup-sampling so that the resolution of the neighboring blocks, which aredecoded by the decoding unit 220, coincides with that of the upper layerwhen the resolutions do not coincide with each other.

The subtractor 230 creates differential neighboring blocks bysubtracting signals, which are directly input from the decoding unit 220or input through the up-sampler 225, from signals that are input fromthe decoding unit 210.

The differential predicted block creation unit 235 performs intraprediction based on the mode information, which is transferred from theentropy decoding unit 205, using the differential neighboring blocks. Asa result, a differential predicted block is created.

The adder 240 creates a predicted block by adding a signal that isoutput from the intra decoding unit 215, that is, the reconstructedcorresponding block (intra block), and the differential predicted block.

Meanwhile, the data on the current block, which are output from theentropy decoding unit 205, are input to an inverse quantization unit245, and an inverse quantization unit 245 inversely quantizes the dataon the current block. This inverse quantization process is the inverseof the quantization process that is performed in the quantization unit175 of the video encoder 100, 300 or 400, and is the process of findinga quantized coefficient that matches a value that is represented by acertain index.

The inverse transform unit 250 reconstructs a residual block byinversely transforming the inverse quantization results. The inversetransform is the inverse of the transform process that is performed inthe transform unit 170 of the video encoder 100, 300, or 400. If thetransform process is DCT, the inverse transform is inverse DCT; if thetransform process is wavelet transform, the inverse transform is inversewavelet transform.

Finally, an adder 255 reconstructs a current block by adding the createdpredicted block and the reconstructed residual block.

The elements of FIGS. 6, 9, 10 and 14 may be software, or hardware, suchas Field-Programmable Gate Arrays (FPGAs) or Application-SpecificIntegrated Circuits (ASICs). However, the elements are not limited tosoftware or hardware. The elements may be constructed to reside in anaddressable recording medium, or to drive one or more processors. Thefunction of each element may be implemented using subdivided elements,and the function may be implemented using a single element that iscomposed of a plurality of sub-elements and performs a specificfunction.

In accordance with the present invention, the advantages of intra BLprediction and the advantages of intra prediction are all taken intoaccount, so that the efficiency of video coding can be improved.

In accordance with the present invention, mode information can be moreefficiently displayed.

Although the exemplary embodiments of the present invention have beendisclosed for illustrative purposes, those skilled in the art willappreciate that various modifications, additions and substitutions arepossible, without departing from the scope and spirit of the inventionas disclosed in the accompanying claims.

1. A method of efficiently predicting a multi-layer based video frame,the method comprising: reconstructing an intra block of a lower layerusing previously reconstructed first neighboring blocks of the intrablock; subtracting the first neighboring blocks from previously storedsecond neighboring blocks of an upper layer corresponding to the firstneighboring blocks; creating a differential predicted block based on anintra prediction mode by performing intra prediction using virtualdifferential neighboring blocks that are created as a result of thesubtracting the first neighboring blocks from the previously storedsecond neighboring blocks; adding the differential predicted block andthe reconstructed intra block; and subtracting a predicted block, whichis created as a result of the adding the differential predicted blockand the reconstructed intra block, from a block of the upper layercorresponding to the intra block.
 2. The method as set forth in claim 1,further comprising up-sampling the first neighboring blocks before thesubtracting the first neighboring blocks from the previously storedsecond neighboring blocks, if a resolution of the upper layer does notcoincide with a resolution of the lower layer.
 3. The method as setforth in claim 2, further comprising up-sampling the reconstructed intrablock before the adding the differential predicted block and thereconstructed intra block, if the resolution of the upper layer does notcoincide with the resolution of the lower layer.
 4. The method as setforth in claim 1, wherein the intra prediction mode comprises nine modesincluding eight directional modes and one DC mode.
 5. The method as setforth in claim 1, wherein the intra prediction mode is selected fromamong nine modes including eight directional modes and one DC mode. 6.The method as set forth in claim 5, wherein the intra prediction mode isselected by adding nine differential prediction blocks based on the ninemodes to the intra block, and selecting one, which has a minimum imagedifference with respect to a block of the upper layer corresponding tothe intra block, from among nine predicted block candidates that areacquired by the addition.
 7. The method as set forth in claim 5, whereinthe selection of the mode is performed by adding nine differentialprediction blocks based on the nine modes to the intra block, applying aRate-Distortion (R-D) cost function to nine encoded differentialpredicted block candidates that are acquired by encoding nine predictedblock candidates acquired as a result of the adding the ninedifferential prediction blocks to the nine encoded differentialpredicted block candidates, and selecting one, which minimizes the R-Dcost function, from among the nine differential predicted blockcandidates.
 8. The method as set forth in claim 5, wherein the selectedintra prediction mode is identical to an optimal intra prediction modethat is used for reconstructing the intra block of the lower layer. 9.The method as set forth in claim 1, wherein the intra block and thefirst neighboring blocks each have a 4×4 size.
 10. The method as setforth in claim 1, wherein the first neighboring blocks comprise a leftblock, an upper left block, an upper block and an upper right block ofthe intra block, and the second neighboring blocks comprise a leftblock, an upper left block, an upper block and an upper right block ofthe block of the upper layer that corresponds to the intra block.
 11. Amethod of encoding a multi-layer based video frame, the methodcomprising: reconstructing an intra block of a lower layer usingpreviously reconstructed first neighboring blocks of the intra block;subtracting the first neighboring blocks from previously stored secondneighboring blocks of an upper layer corresponding to the firstneighboring blocks; creating a differential predicted block based on anintra prediction mode by performing intra prediction using virtualdifferential neighboring blocks that are created as a result of thesubtracting the first neighboring blocks from the previously storedsecond neighboring blocks; adding the differential predicted block andthe reconstructed intra block; subtracting a predicted block, which iscreated as a result of the adding the differential predicted block andthe reconstructed intra block, from a block of the upper layercorresponding to the intra block; and encoding a residual block that iscreated as a result of the subtracting the predicted block from theblock of the upper layer corresponding to the intra block.
 12. Themethod as set forth in claim 11, wherein the intra prediction mode isselected from among nine modes including eight directional modes and oneDC mode.
 13. The method as set forth in claim 12, wherein the intraprediction mode is selected by adding nine differential predictionblocks based on the nine modes to the intra block, and selecting one,which has a minimum image difference with respect to a block of theupper layer corresponding to the intra block, from among nine predictedblock candidates that are acquired by the adding the differentialpredicted block and the reconstructed intra block.
 14. The method as setforth in claim 12, wherein the selection of the mode is performed byadding nine differential prediction blocks based on the nine modes tothe intra block, applying an R-D cost function to nine encodeddifferential predicted block candidates that are acquired by encodingnine predicted block candidates acquired as a result of the adding thenine differential prediction blocks to the nine encoded differentialpredicted block candidates, and selecting one, which minimizes the R-Dcost function, from among the nine differential predicted blockcandidates.
 15. The method as set forth in claim 12, wherein theselected intra prediction mode is identical to an optimal intraprediction mode that is used for reconstructing the intra block of thelower layer.
 16. The method as set forth in claim 13, further comprisingacquiring a directional difference between the selected mode and theoptimal intra prediction mode that is used for reconstructing the intrablock of the lower layer.
 17. A method of decoding a multi-layer basedvideo frame, the method comprising: reconstructing an intra block of alower layer using previously reconstructed first neighboring blocks ofthe intra block; subtracting the first neighboring blocks frompreviously stored second neighboring blocks of an upper layercorresponding to the first neighboring blocks; creating a differentialpredicted block based on an intra prediction mode by performing intraprediction using virtual differential neighboring blocks that arecreated as a result of the subtracting the first neighboring blocks fromthe previously stored second neighboring blocks; adding the differentialpredicted block and the reconstructed intra block; reconstructing ablock of the upper layer corresponding to the intra block; and addingthe reconstructed block of the upper layer and a predicted block that isacquired as a result of the adding the differential predicted block andthe reconstructed intra block.
 18. The method as set forth in claim 17,wherein the intra prediction mode is one of nine modes, including eightdirectional modes and one DC mode, which is transferred from a videoencoder.
 19. The method as set forth in claim 18, wherein thetransferred mode is identical to an optimal intra prediction mode thatis used for reconstructing the intra block of the lower layer.
 20. Themethod as set forth in claim 17, wherein the intra block and the firstneighboring blocks each have a 4×4 size.
 21. The method as set forth inclaim 17, wherein the first neighboring blocks comprise a left block, anupper left block, an upper block and an upper right block of the intrablock, and the second neighboring blocks comprise a left block, an upperleft block, an upper block and an upper right block of the block of theupper layer that corresponds to the intra block.
 22. A multi-layer basedvideo encoder comprising: means for reconstructing an intra block of alower layer using previously reconstructed first neighboring blocks ofthe intra block; means for subtracting the first neighboring blocks frompreviously stored second neighboring blocks of an upper layercorresponding to the first neighboring blocks; means for creating adifferential predicted block based on an intra prediction mode byperforming intra prediction using virtual differential neighboringblocks that are created as a result of the subtracting the firstneighboring blocks from the previously stored second neighboring blocks;means for adding the differential predicted block and the reconstructedintra block; means for subtracting a predicted block, which is createdas a result of the adding the differential predicted block and thereconstructed intra block, from a block of the upper layer correspondingto the intra block.
 23. A multi-layer based video encoder comprising:means for reconstructing an intra block of a lower layer usingpreviously reconstructed first neighboring blocks of the intra block;means for subtracting the first neighboring blocks from previouslystored second neighboring blocks of an upper layer corresponding to thefirst neighboring blocks; means for creating a differential predictedblock based on a predetermined intra prediction mode by performing intraprediction using virtual differential neighboring blocks that arecreated as a result of the subtracting the first neighboring blocks fromthe previously stored second neighboring blocks; means for adding thedifferential predicted block and the reconstructed intra block; meansfor reconstructing a block of the upper layer corresponding to the intrablock; and means for adding the reconstructed block of the upper layerand a predicted block that is acquired as a result of the adding thedifferential predicted block and the reconstructed intra block.
 24. Astorage medium storing a program for executing a method of efficientlypredicting a multi-layer based video frame, the method comprising:reconstructing an intra block of a lower layer using previouslyreconstructed first neighboring blocks of the intra block; subtractingthe first neighboring blocks from previously stored second neighboringblocks of an upper layer corresponding to the first neighboring blocks;creating a differential predicted block based on a predetermined intraprediction mode by performing intra prediction using virtualdifferential neighboring blocks that are created as a result of thesubtracting the first neighboring blocks from the previously storedsecond neighboring blocks; adding the differential predicted block andthe reconstructed intra block; and subtracting a predicted block, whichis created as a result of the adding the differential predicted blockand the reconstructed intra block, from a block of the upper layercorresponding to the intra block.
 25. A storage medium storing a programfor executing a method of decoding a multi-layer based video frame, themethod comprising: reconstructing an intra block of a lower layer usingpreviously reconstructed first neighboring blocks of the intra block;subtracting the first neighboring blocks from previously stored secondneighboring blocks of an upper layer corresponding to the firstneighboring blocks; creating a differential predicted block based on anintra prediction mode by performing intra prediction using virtualdifferential neighboring blocks that are created as a result of thesubtracting the first neighboring blocks from the previously storedsecond neighboring blocks; adding the differential predicted block andthe reconstructed intra block; reconstructing a block of the upper layercorresponding to the intra block; and adding the reconstructed block ofthe upper layer and a predicted block that is acquired as a result ofthe adding the differential predicted block and the reconstructed intrablock.